1. Field of the Invention
The present invention relates to immersive video systems, and specifically to a system and method for displaying immersive videos.
2. Discussion of the Related Art
Immersive videos are moving pictures that in some sense surround a user and allows the user to xe2x80x9clookxe2x80x9d around at the content of the picture. Ideally, a user of the immersive video system can view the environment at any angle or elevation. A display system shows part of the environment map as defined by the user or relative to azimuth and elevation of the view selected by the user. Immersive videos can be created using environment mapping, which involves capturing the surroundings or environment of a theoretical viewer and rendering those surroundings into an environment map.
Current implementations of immersive video involve proprietary display systems running on specialized machines. These proprietary display systems inhibit compatibility between different immersive video formats. Furthermore, the use of specialized machines inhibits portability of different immersive video formats. Types of specialized machines include video game systems with advanced display systems and high end computers having large amounts of random access memory (RAM) and fast processors.
FIG. 1A is a representation of a 360xc2x0 immersive picture P_1, i.e. an environment map. The entire field of view in immersive picture P_1 shows a tree TREE1, a house portion HOUSE1_A, a house portion HOUSE1_B, and a full house HOUSE2. Because memory is arranged in a two-dimensional array, immersive picture P_1 is stored as a two-dimensional array in memory. Thus, the data along edge E1 is not directly correlated to the data from edge E2. As a result, house portions HOUSE1_A and HOUSE1_B, which in the environment of a centrally located theoretical viewer (not shown) are joined into a full house HOUSE_1, are instead separated when immersive picture P_1 is stored in memory. Immersive pictures, such as 360xc2x0 immersive picture P_1, should represent a three-dimensional (e.g. cylindrical) space. As a result, in displaying immersive picture P_1, the two-dimensional representation of FIG. 1A must be converted to a three-dimensional representation.
FIG. 1B is a cylindrical representation of immersive picture P_1 of FIG. 1A. Seam S_1 is formed from joining edges E1 and E2 together to form this cylindrical representation from the two-dimensional representation of immersive picture P_1 shown in FIG. 1A. When edges E1 and E2 are joined as shown, house portions HOUSE1_A and HOUSE1_B are joined into full house HOUSE_1. Thus, seam S_1 runs through full house HOUSE1 and is the dividing line between the house portion HOUSE1_A and the house portion HOUSE1_B. Tree TREE1, located on the door side of house portion HOUSE1_B, is also shown.
FIG. 1C is a conceptual cylindrical representation of the 360xc2x0 immersive picture P_1 of FIG. 1A. The contents of immersive picture P_1 are omitted for clarity. This conceptual cylindrical representation indicates the perception of a theoretical viewer looking at immersive picture P_1 from the vantage point of a location VIEWPOINT, located within the cylinder formed by immersive picture P_1. Immersive picture P_1 is a 360xc2x0 immersive picture having a first edge E1 and a second edge E2. Similarly to FIG. 1B, seam S_1 results from the joining of the two-dimensional representation (FIG. 1A) edges E1 and E2 in the cylindrical representation.
A view window 101 represents the portion of immersive picture P_1 visible to the user at location VIEWPOINT. View window 101 is centered at the origin of a three dimensional space having x, y, and z coordinates, where z (not shown) is perpendicular to the plane of the page. Similarly, the environment surrounding the user located at the location VIEWPOINT is represented by the cylindrical representation of immersive picture P_1 that is centered at the location VIEWPOINT. View window 101 is typically displayed on a display unit for the user of the immersive video system. Thus, only the portion of immersive picture 1 visible to the user, rather than the entire picture content, is displayed, for example, on a television screen.
By moving view window 101 (e.g. left or right) relative to immersive picture P_1, the portion of immersive picture P_1 visible to the user may be changed. This relative movement of view window 101 with respect to immersive picture P_1 is called panning. By moving view window 101 clockwise 360xc2x0, the entire circumference of immersive picture P_1 may be traversed. A cursor 102 within view window 101 is controlled by the user and indicates the desired direction of panning. Cursor 102 is located to the seam S_1 side of view window 101 in FIG. 1C.
FIGS. 1D and 1E are a cylindrical representation of the 360xc2x0 immersive picture P_1 of FIG. 1C rotated clockwise a first and second amount, respectively. Again, the contents of immersive picture P_1 are omitted for clarity. Because cursor 102 is located to the seam S_1 side of view window 101, immersive picture P_1 has panned clockwise with respect to view window 101 from FIG. 1C.
FIG. 1E shows seam S_1 as visible within view window 101. As described above, immersive picture P_1 is stored two-dimensionally in memory, therefore, the data for edge E1 is not directly correlated to the data from edge E2. As a result, when panning across seam S_1, the data from edges E1 and E2 must be joined before being shown to the user on a display as a whole picture. Because real-time picture display systems can""t join images fast enough to display seams, it is preferable not to display seam S_1 in view window 101. It would be desirable to have a method of panning across a picture having seams without real-time seam distortion visibly showing in the view window.
Accordingly, there is a need to deliver an immersive video experience across many different non-specialized platforms while minimizing distortion created by real-time joining of picture seams in the field of view.
In accordance with the present invention, an immersive video system is provided which enables a user to interact with an immersive video on a variety of platforms. To accommodate different types of components found on different platforms, the resolution of the immersive video may be changed to adapt to different amounts of random access memory (RAM) on a given platform.
In one embodiment, a pair of cylindrically defined 360xc2x0 immersive videos are simultaneously played in a standard display software program. These two immersive videos are created such that seams in one video are separated from seams in the second video by at least an amount equal to the length of the view window. The display software program can be chosen such that it is supported by a variety of platforms. For example, choosing Macromedia(trademark) Flash as a display software program allows playback on any platform supporting Flash. A view window associated with the standard display software program defines the portion of the immersive video shown to the viewer. A control mechanism adjusted by the user pans the view window around one of the pair of immersive videos. Panning is the act of moving a point of view in a particular direction (e.g. left of right). Because two immersive videos having different seams are simultaneously played, the view window may select to display a portion of the video without the seam. Thus, if the view window approaches a seam while displaying a portion of a first video, the view window is changed to display a similar portion of a second identical video that has no seam in that location.
In another embodiment, a cylindrically defined immersive video representing an environment map larger than 360xc2x0 (e.g. 420xc2x0) is played in a standard display software program. The overlapping portion of this immersive video (i.e. the portion greater than 360xc2x0) is used to avoid displaying picture seams (or picture edges) to the user.